%
% Section for Future Research
%

% Main section title
\section{FutureResearch}
\label{sec:FutureResearch}

% Subsection title
\subsection{Overview}

ChronoSearch was evaluated on the basis of precision and recall in terms of the event descriptions that were returned. The precision of the system was greatly increased by the three removal techniques utilized: removing bad sentences that did not adhere to typical word lengths, eliminating duplicate events by calculating their cosine similarity, and by removing events that occurred temporally close to each other that had a high verb similarity. An attempt was made to improve the recall of events in ChronoSearch by correlating event importance to the search engine result count returned for that event. As described previously as a lesson learned, this technique for determining the importance of an event did not prove useful. To improve the output of ChronoSearch, future work could be done to increase the precision and recall of the events returned by the system. 

% Subsection title
\subsection{Improving Recall}

One technique for further increasing the recall of the system is to handle relative dates in addition to absolute dates. Recall, in ChronoSearch, an event description is defined as a sentence that contains the literal entity being searched for as well as an absolute date. Event descriptions were in part identified in the system by the use of several regular expressions tailored to match various absolute date types. For instance, one regular expression is utilized in order to match absolute dates of the following form: Monthname dd, yyyy. Another regular expression is created to match absolute dates as: yyyy-mm-dd. It is through the use of several complex regular expressions that ChronoSearch is able to locate and make use of many different forms of absolute dates. 
The technique of using regular expressions would be expanded to also find relative dates within the candidate web pages. Several forms of relative date types exist, such as the following examples: a month ago, last year, yesterday, 2 hours later, 3 years later, and so on. Much like absolute date regular expressions, others could be created to match relative dates in the same fashion. Another way to recognize relative dates would be to make use of wordlists or lists of known word combinations that comprise relative date descriptors. For instance, a wordlist could exist that matches the following: yesterday, last night, last week, next year, etc. The combination of regular expressions and known wordlists would be utilized to identify instances of relative dates within the source content. 

Finding the relative dates is the first part to being able to handle relative dates in ChronoSearch. Once the relative dates have been identified, they then need to be resolved to absolute dates. The idea behind this technique is to resolve relative dates based on the nearest, or last seen absolute date. The methodology behind this concept is that web content is often presented in a chronological order. So, it is likely that a relative date refers to some amount of time relative to the last previously mentioned absolute date. This technique works by finding the first absolute date mentioned in the web page content, then pushing this absolute date onto a LIFO (last-in-first-out) stack. Then, if the next date seen is a relative date, it is resolved with respect to the top absolute date in the stack. If the next date happened to be another absolute date, it would just get pushed onto the stack. This method continues as each date encountered is resolved to an absolute date. Therefore, each time some sort of date information is mentioned, the event can be tied to that date, whether it was absolute or relative. This technique would improve the recall of the events retrieved from the input web pages in the ChronoSearch system.

Another technique aimed at increasing the recall of ChronoSearch is to resolve pronouns to their actual nouns in order to increase the amount of entity information extracted. This approach aims at retrieving more sentences that mention the entity indirectly, rather than using the entity's literal name. A POS (part-of-speech) tagger could be used, or a static list of known pronouns could be utilized to locate pronouns in the source content. This technique would work much like the aforementioned mechanism by which relative dates could be resolved. Pronouns that describe the literal entity would be correlated to the entity being searched and those sentences would be utilized by ChronoSearch. 

An example demonstrating how the recall of the results could be improved is demonstrated by the following text:
An example demonstrating how the recall of the results could be improved is demonstrated by the following text:
\begin{quote}\textit{"\underline{Jobs} became de facto chief after then-CEO Gil Amelio was ousted in \underline{July 1997}. \underline{He} was formally named interim chief executive in \underline{September}."\cite{wiki:Online}}\end{quote} 

From the example demonstrated, one can see that the literal entity being searched (Steve Jobs) is contained in the first sentence as well as an absolute date, namely July 1997. The first sentence would be successfully extracted by the current implementation of ChronoSearch while the second would not be extracted. The second sentence would not be extracted because it contains both a pronoun and a relative date. The approaches described in this section could be utilized to resolve the pronoun, “he”, and the relative date, “September” to their respective absolute meanings “Steve Jobs” and "September, 1997". So, by utilizing the two methods to increase recall as described, the second sentence in the previous example text would be retrieved as an event related to Steve Jobs in September of 1997. 

% Subsection title
\subsection{Improving Precision}

In order to improve the precision of the system, one approach is to eliminate event descriptions that do not actually describe an event. The idea behind this concept is to associate events with verbs. Therefore, the assumption is that if a sentence does not contain a verb, then it does not describe an event. A POS tagger could be used in order to identify verbs within sentences retrieved from web pages. 

With this approach, in order for a sentence to be returned by ChronoSearch, it would need to contain the literal entity or a pronoun mapped to that entity, some sort of date information, and a verb. This technique builds slightly on the current requirements for an event description that is returned by the system by imposing  a rule that a verb must also be contained in the sentence. An example sentence that would be thrown out of the system is demonstrated as follows: \begin{quote}\textit{Farewell Steve Jobs 06 Oct 2011.}\end{quote}
The aforementioned sentence was an actual result found when running the ChronoSearch system. One can see that the sentence does not contain a verb and would be removed from the output list of event descriptions pertaining to Steve Jobs.

Finding verbs within sentences sounds straightforward when using a POS tagger, however, there are difficulties imposed by the data returned from web pages. One difficulty observed when evaluating ChronoSearch was that the NLTK POS tagger failed to identify verbs in some sentences due to the fact that every word was capitalized. This verb similarity approach was not utilized by ChronoSearch because this failure along with others produced an unacceptable amount of false positives.  In an attempt to solve this issue, all capital letters were changed to lowercase except for the first word of every sentence. However this fix caused the POS tagger to fail miserably because without capital letters in the sentence it was unable to accurately detect proper nouns which caused a ripple effect. A large part of the problem is the vast amount of variability of content that resides on the web and the openness of the web. Anyone can post almost any kind of content without being subjected to rules of grammar or syntax. This openness allows for large variations in the correctness of sentences that are retrieved and operated on by ChronoSearch. Resolving verbs in a given sentence is a more difficult problem than just utilizing a POS tagger, and more research would need to take place in order to be able to properly identify all verbs in given sentences. 
